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© System and method for automatic optical inspection. 



© An optical inspection system for inspecting faults on circuit boards like printed circuit boards and wire 
scribed circuit boards is disclosed. A video camera (14) for receiving desired images of hole sites on a wire 
circuit board (18) is provided. The camera converts each image to electrical video signals. A frame grabber (12) 
converts electrical video signals to a plurality of pixels and stores information representing color element values 
contained in each of said pixels. A panel feature detector receives the color element values of the pixel that has 
to be analyzed and provides a wire indication when the color element value of the pixel corresponds to the wire 
feature. A fault detector circuit receives the wire indication from the panel feature detector corresponding to 
predetermined pixels associated with the hole site for determining faults on the circuit board. 
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This invention relates to an automatic optical inspection system, and more specifically to an optical 
inspection system for inspecting faults in high density and high volume discrete wiring panels or wire 
scribed circuit boards. 

In recent times, the electronics industry has moved towards more compact integrated circuit packages 
5 with surface-mounted terminals. The density of wiring patterns have tremendously increased with the recent 
past. Therefore, many circuit board applications today utilize discrete wiring technology which includes 
insulated wires crossing over each other. Consequently, signal paths can run in X, Y and non-axial 
directions on the same substrate. 

Accordingly, insulated wire is scribed into an insulating substrate by feeding a continuous strand of wire 
w onto the surface of the substrate while simultaneously affixing the wire and cutting it at predetermined 
interconnection points. Thus, a wire image of a predetermined interconnect pattern results, which includes 
discrete or discontinuous wire pieces affixed to the substrate. A wire scribed circuit board is described in 
US-A-3,674,914. 

Interconnection points exist at the ends of wires and at certain intermediate points along wires in the 
75 form of metal plated holes which provide conductive paths to the terminals of external components. 

The locations and the sizes of the holes are predetermined and replicable for each wire scribed board. 
Generally, it is preferred to first apply the insulated, preformed wires to the board surface along a pre- 
programmed path and to then drill or lase the board at hole sites which are those locations at which 
terminals are to be located. Plating material may then enter the holes, connecting the wire terminations to 
20 external pads. 

Once a discrete wiring board has been scribed, it is necessary to inspect the board for any wiring 
errors. Conventionally, a human operator may inspect the board visually. However, for a typical board there 
is a multitude of wires, which renders visual inspection highly inefficient and errorprone. The problem is 
critical at interconnection sites. Since the scribed wires at drilling points are stripped for later connections, 
25 any scribing error may cause a "short" or an "open" at an interconnection site. 

Automatic inspection systems have been quite useful in detecting faults compared to visual inspection. 
Conventionally, automatic optical inspection machines have been implemented based on two fundamental 
methods. The first method is the design rule approach. A board is scanned in search of minimum design 
rule violations which are detectable within the span of a very small window. Minimum trace width and trace 
30 separation violations, as well as minuscule defects like scratches are targets of the design rule check. 

The second method is the reference comparison approach. The reference method compares the test 
board, pixel by pixel to a representation of a "golden board" stored in a memory. 

Prevalent automatic optical inspection architecture includes panel illumination with incoherent light, and 
image detection with a monochrome charge coupled device (CCD) array. For such systems, inspection of 
35 artwork and hole sites is provided by back-lighting, and inspection of conductor against substrate is 
accomplished by top-lighting. In back-lighting, the illumination ' and detection hardware are located on 
opposite sides of the article under inspection. In top-lighting, the detection hardware and illumination emitter 
are located on the same side. 

Automatic optical inspection with top-lighting is based on illuminating the board and detecting different 
40 intensities of reflected light. Thus, conductor and substrate may be recognized by the amount of their 
reflected light. For example, bright shiny copper can be easily differentiated from dull dark substrate. 

However, inspection of copper oxide is not as easy. A difficult inspection task is detection of faults at 
solder sites. Difficulties arise because solder has an irregularly shaped three dimensional surface and is 
very shiny. These qualities cause it to reflect light in an irregular manner. 
45 Inspection of discrete wiring boards has proven to be very difficult as well. Like solder, wire has a 
curved three dimensional surface which reflects light irregularly. The way the wire was scribed, whether 
adhesive has oozed over it, whether it has suffered any abrasion and the existence of nearby crossovers, 
are all determinants as to the manner in which it reflects light. Furthermore, a discrete wiring panel prior to 
encapsulation is strewn with topographic features other than wire. Bumps, scratches, troughs and ridges in 
so the soft shiny substrate can result in bright reflections which act as wire impostors to conventional 
automatic optical inspection systems. 

For example, when viewing a discrete wiring board illuminated from the top, the distribution of wire and 
substrate brightness fall into two broad overlapping domains. In the region of overlap, analysis based solely 
on light intensity is subject to potential confusion as to the type of material present. Consideration of spatial 
55 structure can help to reduce ambiguity, but it is computationally expensive, and often still not decisive. 
Thus, it becomes necessary to treat marginal decisions conservatively, to err in favor of rejecting good 
points on a panel rather than authorizing bad points. Such rejection criteria are a further disadvantage with 
conventional automatic inspection systems. 
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The task of differentiating wire from substrate becomes even more difficult when the color of the 
scribed wire is similar to the color of the substrate. In this event, the wire and substrate brightness cam 
overlap substantially, causing many false rejections. 

Thus, there is a need to inspect highly dense circuit boards with high efficiency and high reliability, 
5 without the disadvantages described hereinabove. 

It is an object of this invention to provide a reliable system for automatic optical inspection of circuit 
boards generally. 

It is also an object of this invention to provide a system and a method for automatic optical inspection 
of discrete wire circuit boards including printed circuit subcomponents of discrete wire circuit boards. 
w It is a further object of this invention to provide a color based automatic optical inspection system for 
inspecting circuit boards including printed circuit boards and discrete wire circuit boards. 

It is also an object of this invention to provide a color based automatic optical inspection system with 
the ability to distinguish substrate and wires having similar colors. 

Another object of this invention is to provide an automatic optical inspection system capable of 
75 measuring panel stretch and shrinkage as indicated by shifts in the position of registration. 

A still further object of this invention is to provide an automatic optical inspection system capable of 
compensating for any errors arising from incorrectly aligning a circuit board on the inspection X-Y table. 

These and other objects are achieved by the optical inspection system as defined in claims 1,15 and 
29 as well as the method defined in claim 19. In accordance with the invention, a video camera is disposed 
20 above the wiring board to be inspected. The video camera receives images of the board and sends the 
information to an analyzer which conducts the analysis portion of the system. The system uses all color 
information received from the video camera, and works directly with "Red", "Green" and "Blue" (RGB) 
look-up tables. 

In another embodiment of this invention, the image data in RGB format is converted to image data in 
25 "Hue", "Saturation" and "Intensity" (HSI) format. The system then uses the hue component of the HSI data 
but ignores the saturation and intensity components, in order to produce the highest possible processing 
speed. 

In another embodiment, instead of took-up tables, neural network technology operating directly on RGB 
data may be used to differentiate between wire and substrate. 
30 The system establishes a three dimensional look-up table representing RGB space. A large fraction of 
all RGB values belonging exclusively to wire features then address set bits in the wire look-up table. All 
RGB values which could belong to a substrate feature must address a zero bit in the wire look-up table. 
Conversely, a large fraction of all RGB values belonging exclusively to substrate features would then 
address set bits in the substrate look-up table. 
35 The system is first operated in a training mode to sample wire and substrate areas and therefore set up 
the corresponding wire and substrate look-up table. Each time a sample is made, the areas which the 
system has been trained to associate with substrate turn green and the areas which the system has been 
trained to associate with wire turns bright red. 

During actual inspection, interconnection sites are checked for open and short conditions. A reference 
40 file contains information describing the correct wiring at each interconnection site, including the hole type, 
and the angle of entry and departure of each wee. If an inspected interconnection site appears inconsistent 
with the reference file, a fault is deemed detected. 

Figure 1 illustrates one embodiment of the invention for automatic optical inspection of circuit boards. 
Figures 2A-2C illustrate an example of a discrete wiring panel to be inspected in accordance with the 
45 invention. 

Figure 3 illustrates a functional block diagram of an embodiment of the invention. 

Figures 4A and 4B represent an example of a three dimensional RGB space for wire and substrate, 
respectively. 

Figure 5 illustrates the structure of wire and substrate look-up tables according to one embodiment of 
50 the invention. 

Figures 6A represents an image with a small open circuit reticle and a large short circuit reticle 
imposed on a hole site. 

Figure 6B illustrates an example of the contents stored in the open/short scan table 96 of Figure 3. 
Figure 7 illustrates an example of pixel/bin map 90. 
55 Figures 8A and 8B illustrate diagrams of the processed histogram provided by the processed histogram 
memory. 

Figure 9 illustrates a flow chart of operations of cluster analyzer 110. 
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Figures 10A and 10B illustrate a job file which contains pertinent information about all the hole sites to 
be inspected on a panel. 

Figure 10C illustrates ah example of entry and exit points at a hole site. 
Figure 11 A illustrates a panel feature detector according to one embodiment of the invention. 
5 Figure 11B illustrates a single node as shown in Fig. 11 A. 

Figure 12A illustrates another panel feature detector according to one embodiment of the invention. 
Figures 12B and 12C illustrate hue threshold parameters T1 and T2. 

Figures 13A-13F illustrate some examples of good and defective wire crossings at hole sites. 
Figure 1 illustrates an embodiment of the invention for automatic optical inspection of circuit boards. 
10 Computer 10 controls the operation of the system and may be an inexpensive personal computer with 
preferably a 16 bit or 32 bit microprocessor. 

A frame grabber 12 is connected to the computer and may occupy one slot in the computer backplane. 
Frame grabber 12 may be one of the readily commercially available components. 

Frame grabber 12 is coupled to an RGB video camera 14. The video camera is disposed above a 
/5 moveable X-Y table 16 which can be moved along the X and Y axes. On top of table 16, a discrete wiring 
panel 18 is disposed for inspection. A light illuminator 20 provides incident light on the discrete wiring panel 
18. A motion controller 22 is connected to the input/output of computer 10 and X-Y table 16. 

An RGB video monitor 24 is connected to frame grabber 12 and provides a visual image of the discrete 
wiring panel site viewed by RGB video camera 14. RGB video camera 14 includes a plurality of charge 
20 coupled device arrays receiving reflected light from the discrete wiring panel 18. Frame grabber 12 receives 
images from the RGB video camera 14 in RGB format. Therefore, each element in the image is represented 
by distinct red, green and blue components. 

Frame grabber 12 converts analog signal representing the image viewed by the RGB video camera into 
a digital two dimensional array. The size of the digital two dimensional array in the frame grabber is 512 x 
25 480 pixels in US and 512 x 512 pixels in Europe. There is also a fixed correspondence between positions 
on the photoreceptor chip of the RGB video camera 14 and the two dimensional array of frame grabber 12. 
Therefore, every pixel location in the two dimensional digital array of frame grabber 12 corresponds to a 
unique position in the image detected by the RGB camera. 

Thus, the data provided by frame grabber 12 is readily available for the computer's use in a convenient 
30 format. 

Motion controller 22 receives instructions from computer 10 to move X-Y table 16 to a desired location. 
Thus, all points in the discrete wiring panel 18 may be inspected by RGB video camera 14. 

Figures 2A-2C illustrate an example of a discrete wiring panel to be inspected in accordance with the 
invention. A plurality of scribed wires 28 are disposed on the panel 18 and pass through or end at 
35 interconnection sites 30. As illustrated, some of the interconnection sites 30 have no wires crossing through 
them. Such sites may be drilled later to provide through holes used for power and ground terminations or 
for mechanical purposes. Interconnection sites 30 are hole sites that will be drilled in a later stage of 
discrete wiring panel manufacturing. 

Each hole site 30 on the discrete wiring panel 18 has a predetermined location and a corresponding 
40 predetermined diameter. 

Each discrete wiring panel 18 has four registration coupons 29, one at each corner of the panel. Panel 
of different sizes have coupons in different nominal positions, the coordinates of which are stored in a file 
accessible to computer 10. The registration coupons provide reference points so that the exact locations of 
predetermined hole sites may be found on each panel with accuracy. 
45 In order to enable the system to distinguish between a wire and a substrate, a training procedure must 

first be initiated. The system inspection capabilities hinge on a three dimensional look-up table 34 (Fig. 1) 
which represents RGB space. 

Accordingly, Figure 3 illustrates a functional block diagram of an embodiment of the invention. As 
described above and illustrated in Figure 1, frame grabber 12 receives image information from RGB camera 
so 14. Microprocessor 10 is coupled to a console monitor 50 and a keyboard 52. Thus, an interface between 
an operator inspecting the board and the system may be provided by utilizing the keyboard and a menu 
displayed on monitor 50. Microprocessor 10 also monitors and controls an analyzer 40. As it will be 
appreciated by those skilled in the art, analyzer 40 may be implemented either in hardware or software. 

Analyzer 40 includes various subsystems for determining the status of wires at hole sites 30. 

55 
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Panel Feature Detector 

One subsystem of analyzer 40 is a panel feature detector 80 capable of determining whether a pixel 
provided by frame grabber 12 corresponds to a wire or a substrate of panel 18 under inspection. Panel 

5 feature detector 80 includes a wire look-up table 82, a substrate look-up table 84 and an inhibitor 86. Both, 
wire look-up table 82 and substrate look-up table 84 are addressed by a large fraction of the Red ("R"), 
Green ("G") and Blue ("B") values of pixels provided by frame grabber 12. 

Thus, look-up table 82 and 84 are a representation of a three dimensional "R" "G" "B" space. 

Figures 4A and 4B represent an example of a three dimensional RGB space 150 and 152 for wire and 

10 substrate, respectively. Thus, the wire bit distribution 154 in Figure 4A represents points in RGB space 
which correspond to wire features, and the substrate bit distribution in Figure 4B represents points in RGB 
space which correspond to substrate features. Therefore, all the RGB values that belong to a substrate 
feature will address a zero bit in the wire RGB space 150 and all the RGB values that belong to a wire 
feature will address a zero bit in the substrate RGB space 152. 

15 Accordingly, every combination of RGB values would refer to a spot in the wire or substrate RGB 

space. The wire and substrate look-up tables could then be addressed by an "R" and a "G" and a "B" 
value and the corresponding data can be set to "1" or "0" depending on whether the RGB value belongs to 
wire or substrate. However, a "1" or a "0" may occupy eight bits in a conventional computer or 
microprocessor environment. In one embodiment of the invention, the wire and substrate look-up tables 

20 have been set such that a "1" or a "0" value may occupy one bit only. 

Figure 5 illustrates the structure of wire and substrate look-up tables 82 and 84 according to one 
embodiment of the invention. Each dimension of the table corresponding to Red, Green or Blue colors has 
a resolution of 6 bits, for a total table size of 262144 bits. This translates into 32768 bytes. Since the raw 
data available from frame grabber 12 is provided at 8 bits per RGB component, this represents a four-fold 

25 compression in each dimension, six bits per dimension, however, has proven to be a satisfactory balance 
between dynamic range and memory conservation. 

Table 160 in Figure 5 corresponds to tables 82 or 84 of Figure 3. Every consecutive 8 bytes span the 
limits of the blue axis. Each byte contains eight bits, subdividing the blue axis into a total of 64 discrete 
increments. The green axis also has 64 increments. Adjacent increments along the green axis correspond 

30 to addresses eight bytes apart in the look-up table. The red axis also has 64 increments. Adjacent 
increments along the red axis correspond to addresses 512 bytes apart (8 x 64 = 512) in the look-up table. 

Consequently, once an RGB value is provided by frame grabber 12, the R and G and B components 
may be fused into one 18 bit binary number. The six most significant bits correspond to a Red value and 
the second six most significant bits correspond to a Green value. The six least significant bits correspond to 

35 a Blue value. Thereafter the three least significant bits corresponding to a Blue value are then stripped and 
the remaining 15 bits provide a byte address. Each bit in the addressed byte nay further be addressed by 
the three least significant bits which were stripped. Therefore, any bit in the look-up table can be addressed 
and set to "1" or "0" accordingly. For example, assuming that a wire feature has R = 0, G = 0 and B = 27 or 
(011,011), the bit corresponding to location 162 in table 160 would be set to "1" if table 160 was a wire 

40 look-up table. The three most significant bits of Blue value contribute to a byte address, and the three least 
significant bits of Blue value, point to a bit location in that byte. 

It is then possible to fill up the tables 82 and 84 by setting appropriate bits to a "1 " corresponding to an 
RGB value derived from wire or substrate. 

Inhibitor 86 in Figure 3 monitors RGB values that have a set bit in both the wire and substrate look-up 

45 table and sets the corresponding bit in the wire look-up table 82 back to zero. Thus, any RGB values that 
are in both the wire and substrate look-up tables 82 and 84 are interpreted as substrate. 

In a preferred embodiment of the invention, frame grabber 12, Figure 3, provides data on a row by row 
basis to pixel row butter 88 which retains information of all pixels contained in a row and provides this 
information to panel feature detector 80 on a pixel by pixel basis. 

so Therefore, the input of panel feature detector 80 is an RGB value corresponding to a pixel in a row of 
pixels provided by pixel row buffer 88, and the output of panel feature detector 80 is a bit at line 83 
provided by the wire look-up table 82 indicating whether the pixel corresponds to a wire or not. 

Pixel Retrieve Controller 

55 

Figure 3 illustrates pixel retrieve controller 91 including an open/short scan table and a keepout scan 
table 98. The outputs from open/short scan table 96 and keepout scan table 98 are both coupled to a pixel 
row transfer controller 94 and pixel row scan controller 92. 

5 
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The data stored in the open/short scan table 96 is based in the systems's short and open detection 
criteria at hole sites and the data stored in keepout scan table 98 is based on the criteria that some hole 
sites should remain without any wire crossings. 

It can be appreciated by those skilled in the art that the frame grabber 12 may store the image of a 
5 hole site 30 of Figure 2. The data stored in tables 96 and 98 is explained more in detail in conjunction with 
Figure 6. 

Figure 6A represents a hole site image 162, with a small open-circuit reticle 164 and a large short 
circuit reticle 166 which is an annular ring imposed on the hole site image. It can be appreciated by those 
skilled in the art that the short circuit reticle and the open circuit reticle may be actually displayed on the 

70 video monitor at the location of a hole site. Short circuit detection is based on three geometric rules. The 
rules are applied to the short circuit annular ring 166 which has an inner diameter slightly larger than the 
diameter of holes at inspection sites. The geometric rules are: (a) wires always cross the ring orthogonally; 
(b) the arc length of any wire crossing is bound by an upper limit; and (c) the ring may be crossed no more 
than a predetermined number of times. In a preferred embodiment, the number of allowed crossings is 

15 limited to two. 

The open circuit detection ring 164 is substantially a circular region at the center of a hole site being 
inspected. Open circuit detection discovers a violation when insufficient wire is found within the open circuit 
detection ring 164. 

The inner diameter of the short circuit reticle 166 can change for each new hole size and its width 

20 varies slightly depending on the inner diameter of the short circuit reticle 166. 

The open/short scan table stores for each row the number of pixels from a vertical diameter 165 to the 
outer and inner circles formed by short circuit reticle 166. The open/short scan table also stores for each 
row the number of pixels from the vertical diameter to the circle formed by open circuit reticle 164. 

Thus, for example, for a scan line 168, the open/short scan table stores the number of pixels from the 

25 center of the ring to the outer circle of the short detection annular ring, and the number of pixels from the 
center of the ring to the inner circle of the short detection annular ring and the number of pixels from the 
center of the ring to the diameter of the open detection ring 164. The number of open/short scan lines 
depends on the inner diameter of the short detection annular ring. Generally, the larger the annular ring, the 
open/short scan lines are farther apart in order to keep the number of scan lines from getting too large. 

30 Figure 6B illustrates an example of the contents stored in the open/short scan table 96 of Figure 3. 

Thus, for one particular hole site, for each open/short scan line number, there are three corresponding 
distance parameters in pixel units. For example, for scan line 168, which is the center-most scan line, 
during inspection the outer bound of short detection reticle is 180 pixels, the inner bound of short detection 
reticle is 170 pixels and the bound of open detection reticle is 50 pixels. Similarly, for scan line number 

35 three (3) which is three scan lines away from center-most scan line, the outer bound of short detection 
reticle is 172 pixels, the inner bound of short detection reticle is 160 pixels, and the bound of open 
detection reticle is 42 pixels. Table 96 contains only the open/short scan line numbers that are required to 
be analyzed. There may be more than one open/short scan table for a system with hole diameters of 
various sizes, since the scan line parameters for each hole site are likely to be different. 

40 The open/short scan line information is provided to pixel row scan controller 92 and pixel row transfer 

controller 94, the latter receiving the parameters of the next scan line to be analyzed from open/short scan 
table 96. Pixel row transfer controller then sends the correct number of pixels in the desired scan line to 
pixel row buffer 88 so that the desired scan line can be retrieved from frame grabber 12. 

Pixel row scan controller 92 receives the pixel boundary information of the short and open circuit 

45 reticles from open/short scan table 96 and, based on that information, sends a control signal to pixel row 
buffer 88. Consequently, pixel row buffer 88 sends the corresponding pixel information to wire look-up table 
82 on a pixel by pixel basis. The control information provided by pixel row scan controller 92 is basically 
the address of next pixel to be analyzed in a particular row and column number. The row number being the 
scan line number and the column number being the number of the pixel to be analyzed in that row. 

so Pixel row scan controller 92 also provides the pixel row and column number to a pixel/bin nap 90. The 
operation of pixel/bin map 90 is explained in more detail in conjunction with Figure 6A. It can be 
. appreciated by those skilled in the art that the addresses of pixels in the open and short circuit reticles may 
be stored individually and subsequently color information corresponding to the pixels can be sent to panel 
feature detector 80 based on the addresses of individual pertinent pixels without resort to pixel row buffer 

55 88. 

As illustrated in Figure 6A, the short circuit reticle is divided into many sections or bins 170. According 
to one embodiment of the invention, the short circuit reticle is divided into 256 bins. Each bin 170 has an 
assigned bin number. The pixel/bin map 90 of Figure 3 stores the assigned bin number of each 
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corresponding pixel in the short circuit reticle 166. Figure 7 illustrates an example of pixel/bin map 90. 
Accordingly, for each type of hole site, particular open/short reticle dimensions are utilized by the system. 
Once the appropriate open/short reticle information is loaded in the open/short scan table 96, the data in 
pixel/bin map 90 will be set also. It can be appreciated by referring to Figure 6A that the pixels contained in 
5 each scan line may fall into more than one bin in the short reticle 166. As the scan lines move further away 
from the center-most scan line 168, the number of bins corresponding to the pixels on the same scan line 
increases. 

Figure 6A illustrates one example for determining the bin number assigned to pixels 172 and 174. 
Accordingly, the scan line number corresponding to scan line 169 and the column number corresponding to 
w pixel 172 is derived as the X and Y coordinates of pixel 172. Based on that information, angle 0i 
corresponding to pixel 172, is measured by calculating the arc tangent (y/x). Similarly, angle 0 2 correspond- 
ing to pixel 174 is measured. The corresponding bin number is determined as follows: 



5 bin number = 0X256 

360 

The bin number is scaled to eight bit numbers and is stored as bytes Xi , X 2 , .... X n in pixel/bin map 90 of 
20 Figure 7. 

It can be appreciated by those skilled in the art that the memory arrangement for open/short reticles 
and pixel/bin conversion may be performed in various ways. For example, the open/short scan table 96 can 
store parameters corresponding to every scan line present in frame grabber 12 (Fig. 3). Similarly, pixel/bin 
map 90 may contain bin numbers assigned to each pixel in every scan line. In that event, when certain scan 
25 lines are skipped during analysis of only the pertinent scan lines associated with a particular hole type, then 
a pointer table indicates the location in pixel/bin map where each new scan line begins. 

Shorts Detector 

30 As illustrated in Figure 3, the bin number assigned to each pixel to be analyzed is sent to short detector 
100. 

Short detector 100 includes a bin histogram memory 102 which also receives a signal from wire look-up 
table 82 on line 83. Thus, for each pixel to be analyzed, bin histogram memory 102 receives the bin 
number assigned to that pixel from pixel/bin map 90 and receives a signal indicating whether said pixel 
35 represents a wire or not from wire look-up table 83. Thus, after analyzing all the pixels of a hole site, a bin 
histogram of wire class pixels is formed on bin histogram memory 102. For each wire crossing a hole site, 
there is going to be a peak over a few adjacent corresponding bins in the bin histogram memory 102. 
Violation may thus be detected when the majority of pixels in the bin histogram cannot be attributed to at 
most two peaks. 

40 It can be appreciated by those skilled in the art that the important data in bin histogram memory 102 

may be obscured due to various system noise. Thus, the data in the bin histogram memory 102 may be 
further processed to alleviate the effects of system noise. 

Accordingly, a digital filter 104 receives bin histogram data from bin histogram memory 102. To a first 
approximation, the wire class pixels associated with a wire crossing at the short detection reticle from a 
45 triangular distribution as a function of angular position. 

Digital filter 104 is a moving window averaging filter and its general response is defined by equation (1) 
below: 

so (1) w/2 w/2 

i=l LeJ 

D 

55 

where S is the input signal; R is the output response of filter 104; W is the window width and D is a divisor. 
According to one embodiment of the invention, digital filter 104 is defined as equation (2): 



7 



BNSDOCID:<EP 05941 46A2> 



EP 0 594 146 A2 



V 



(2) 



2 

i = l 



2 

i = l 



5>k-» +3 + S k 



The output of filter 104 is then provided to a processed histogram memory 106 which will contain a 
smoothed version of the histogram formed in bin histogram memory 102. 

w In order to detect the center of each wire crossing, the data contained in processed histogram memory 

106 is provided to a digital filter 107. The center of each wire crossing is necessary information for 
determining whether multiple wire crossings have occurred too close together, violating design rules, and 
whether actual crossing positions deviate sufficiently from expected crossing positions to violate reference 
rules. A wire crossing center is considered to be the location wherein the wire class pixel density after 

75 smoothing reaches a maximum. 

Sometimes, however, wire crossings create double-cusped peaks. In such cases, the peak center could 
be identified as either one of the cusps, rather than the small valley in between. Digital filter 107 is designed 
to prevent this type of error. 

Digital filter 107 is a peak enhancement filter. Its response is proportional to signal amplitude, and 

20 favors triangular peaks. The general response of filter 107 is defined by equation (3) as follows: 



25 



(3) 



w/2 
i = l 



w/2 

s 

i=l 



i x S 



+ (w/2 + 1) S k 
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where S is the input signal provided by memory 106; R is the output response; and W is the window width. 
According to one embodiment of the invention, filter parameters defined by equation (4) are as follows: 

7 7 

(4) ™ ^ 

R k = Zr i x S k+i ^ + la i X S k . i+8 + 8 x S k 

i=l i=l 



the coefficients for this filter being: 
1, 2, 3, 4, 5, 6, 7, 8, 7, 6, 5, 4, 3, 2, 1. 

Figures 8A and 8B illustrate diagrams of the processed histogram provided by processed histogram 
40 memory 106. 

The output of digital filter 107 is coupled to a cluster identifier 108. The cluster identifier determines the 
bin numbers corresponding to the centers of the two highest distributions assumed to be wire crossing. 

Accordingly, the cluster identifier 108 provides the bin numbers corresponding to each of the two peaks 
(248, 250, Figures 8A and 8B) to cluster analyzer 110. Cluster analyzer also receives data from, processed 
45 histogram memory 106. 

Figure 9 illustrates a flow chart of operations of cluster analyzer 110. Accordingly, at step 200, cluster 
analyzer 110 places a triangle 242 at curve 240 having the largest peak in the processed histogram 
received from the processed histogram memory 106. Thereafter, at step 202, the triangle 242 with a height 
"H1" is set to 1.5 times the peak height "PI". At step 204, the triangle base is set to 2 times the window 
so parameter. The window parameter is used to control the size of the triangle 242 base and the filter 107 
windows. A default value of 15 (bins) has been found to be most appropriate to date. Consequently, at step 
206, the area A1 of curve 240 confined within triangle 242 is measured by counting the wire-pixels in that 
area. 

At step 208, cluster analyzer 110 places a triangle 246 at curve 244 having the second largest peak in 
55 the processed histogram received from the processed histogram memory 106. Thereafter, at step 210, a 
triangle 246 with a height "H2" is placed over curve 244, spaced at least one window apart from the 
location 248 of the first peak. Height H2 of triangle 246 is set to 1.5 times the peak height "P2". 
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At step 212, the area A2 of curve 244 confined with triangle 246 is measured by counting the wire 
pixels in that area. 

At step 214, cluster analyzer 110 measures the total area A3 of the entire processed histogram received 
239 by counting all the pixels in all of the histogram bins. 
5 At step 216, the ratio of pixels in area A1 and area A2 over the total number of pixels in area A3 is 

determined. 

At decision step 220, the cluster analyzer 110 determines whether the ratio measured at step 216 is 
equal to or larger than 0.9. If not, the cluster analyzer 110 goes to step 222 and sets an "ERROR" flag. If, 
however, the ratio measured at step 216 is equal to or larger than 0.9, the system goes to decision step 
w 224, since the majority of the pixel counts in the bin histogram memory has been attributed to at most two 
peaks. 

At decision step 224, cluster analyzer 110 determines the distance between the first peak H1 and the 
second peak H2 and converts that distance to a degree angle. If the degree angle between two peaks is 
more than a threshold, the system goes to step 228 to set a "GOOD" flag. In one embodiment of the 
75 invention, the threshold is 22°. If not, the system goes to step 226 to set an "ERROR" flag. Also, the two 
peak positions are compared with reference positions and if the discrepancy is too great, the system 
generates an error. In a preferred embodiment, 45" is an appropriate good/bad threshold. 

Open Detector 

20 

Open detector 120 (Fig. 3) determines whether an open circuit will occur at a hole site when the hole is 
actually drilled. Open detector 120 includes a wire pixel counter 122. The wire pixel counter receives wire 
pixel counts from wire look-up table 82 on line 83. The wire pixel counts are associated with wire pixels 
present in open circuit reticle 164 (Fig. 6A). 

25 Therefore, the pixel row scan controller 92 and pixel row transfer controller 94 receive from open/short 

can table 96 the pixel numbers present in a scan line and confined within the open circuit reticle 164. Thus, 
the pixels to be analyzed are transferred to pixel row buffer 88. Once a pixel being analyzed is attributed to 
wire, the wire pixel counter 122 increases once count. It can be appreciated by those skilled in the art that 
the pixels for opens detection may be derived from a single row transfer which serves opens and shorts 

30 detection. 

After completion of all scan lines having pixels within the open circuit reticle, the wire pixel count is sent 
to a comparator 124. Comparator 124 receives a threshold reference from threshold reference memory 128 
and compares the number of wire pixel counts to threshold reference. If the number of wire pixel counts is 
below the threshold reference, the comparator sends an "ERROR" flag to open counter 126. 

35 

Keepout Detector 

As shown in Fig. 3, keepout detector 130 determines whether there are any illegal wire crossings at an 
unwired hole site. Similar to open detector 120, keepout detector 130 includes a wire pixel counter 132, 

4o providing the wire pixel count of a keepout reticle to a comparator 134. Comparator 2134 then compares 
the wire pixel count provided by wire pixel counter 132 with a threshold reference stored in threshold 
reference memory 138. If the number of wire pixel counts is above the threshold reference, the comparator 
sends an "ERROR" flag to keepout counter 136. 

Keepout detector 130 receives wire pixel count triggers from wire look-up table 82 on line 83. The 

45 location of pixels to be analyzed for keepout detection are stored and provided by keepout scan table 98. 
Keepout scan table provides the scan line number and the number of pixels on each scan line in keepout 
reticle. The information is then utilized by pixel row transfer control 94 and pixel row scan control 92 to 
send appropriate pixels to wire look-up table 82, and consequently, wire pixel counts to wire pixel counter 
132. 

50 

Registration Controller 

Figure 3 also illustrates the registration controller 62. Each panel for inspection has four registration 
coupons one on each corner of the panel. Each position on the panel 18 which needs to be inspected is 
55 addressed by X and Y coordinates. Thus, if the panel on the X-Y table 16 is not aligned accurately, the 
system would not be able to determine a desired position on the panel. Furthermore, due to various 
reasons, panel 18 may become distorted to an extent that even if it is aligned precisely on X-Y table, due to 
distortions, the system would be unable to determine a desired position on the panel. The registration 
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controller 62 according to an embodiment of the invention compensates for such misalignment or 
distortions, To measure the alignment error and dimensional distortion of a panel, the registration controller 
62 determines the position of the four registration coupons. Figure 2 illustrates registration coupons 29 on 
panel 18. Accordingly, each registration coupon has three vertical and horizontal wires. 

5 Registration controller 62 in Figure 3 includes a row profile buffer 64 and a column profile buffer 66, 

each receiving wire pixel counts from wire look-up table 82 on line 83. Each element of the row profile 
buffer 64 provides the sum of wire pixels found in a particular row of an image. Each element of the column 
profile buffer 66 provides the sum of wire pixels found in a particular column of an image. The information 
from row profile buffer 64 and a column profile buffer 66 is provided to a cluster identifier 68 locating all 

io clusters corresponding to wires in the row and column profile. 

If three clusters were found in each buffer, the information derived in cluster identifier 68 is then 
provided to cluster analyzer 70. At cluster analyzer 70, the center of the three clusters in the row profile and 
the center of the three clusters in the column " profile are determined. Thus, the X, Y coordinates of the 
center of each registration coupon can be obtained. 

75 The X, Y coordinates of the center of each registration coupon are then sent to a compensator 72 which 

compares the actual X, Y coordinates of registration coupons with the reference X, Y coordinates of 
registration coupons of an ideal panel. Thereafter, the compensator 72 provides the offset values of X, Y 
coordinates for the panel under inspection, corresponding to where the registration coupons on the panel 
are and where they should be. The offset value includes a translation and a rotation. 

20 An X, Y transformer 58 is coupled to compensator 72. The X, Y transformer 58 provides a transforma- 

tion matrix between the ideal positions of hole sites to be inspected and actual positions of those hole sites 
on the panel. 

Job File 

25 

Figures 3 and 10A and 10B illustrate a job file 54 which contains pertinent information on all the hole 
sites to be inspected on a panel. The first field H of job file 54 designates a hole code 250 based on the 
size of the hole. 

Hole code 250 is a pointer to another record 280 (Fig. 10B) having six fields, 266-276. The six fields, 
30 266-276, designate the reticle sizes associated with each hole code. The first three fields, 266, 268, 270, 
contain reticle information used during automatic inspection. The first three fields reticles define an open 
circuit reticle 266, short circuit reticle 268, and keepout reticle 270. The remaining three fields are the 
reticles used during defect review, and are chosen in accordance with accept/reject criteria used during 
defect review. 

35 The remaining fields of the job file include fields 252 and 254 which represent the X, Y coordinates of 

each hole site to be inspected. Fields 256 and 258 represent the entry and exit vectors of each wire 
crossing a hole site. 

Figure 10C illustrates an example of entry and exit vectors at a hole site. Accordingly, each hole site is 
divided into a few sections. Thus, for example, wire 282 (Fig. 10C) enters the hole site along vector 1, 
40 Ventry = 1 , and exits the hole site along vector 6, Vexit = 6. 

The remaining three fields, 260, 262, 264 (Fig. 10A) provide the defect status information for the 
inspected holes. Accordingly, field 260 is set when a violation after keepout reticle scan is encountered. 
Similarly, fields 262 and 264 are set when violations after short/open reticle scan are encountered. 

As mentioned before, motion system controller 22 (Figures 3 and 1) provides motion control for X-Y 
45 table 16. Motion system controller 22 receives compensated X, Y coordinates from X-Y transformer 58 (Fig. 
3). It can be appreciated by those skilled in the art that whenever a desired hole site needs to be inspected, 
the X, Y coordinates in fields 252 and 254 of job file 54 are provided by X, Y transformer 58 so that the 
actual hole site location on the panel is obtained. 

so Neural Network Feature Detector 

According to another embodiment of the invention, panel feature detector 80 is substituted by another 
panel feature detector based on a neural network. 

Figure 11A illustrates a panel feature detector 300 which receives RGB information from frame grabber 
55 12 (Fig. 3), as explained before. The output of node 302 indicates whether the RGB values of a pixel 
correspond to a wire or not. . 

Panel feature detector 300 is a two hidden-layer fully connected feed forward neural network. The input 
payer contains three nodes 304, 306, 308 with linear transfer functions. Nodes 304, 306, 308 distribute 
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incoming RGB data to the rest of the network. The following layers, including nodes 310-320 and nodes 
322-332, have connections which do not appear as inputs or outputs to the real world, and are thus called 
hidden layers. All layers with the exception of the input layer, have sigmoidal transfer functions. Sigmoidal 
transfer functions, also known as "squashing functions", map a large input domain, typically negative to 
5 positive infinity, into a small output range which can be used in panel feature detector 300 as follows: 

f(X): 1 

-(X) 

10 1 + e 



In general, a squashing function must be continuous, monotonic and differentiate at all points. In a 
continuous-valued artificial neural net implementation with output values ranging from 0,0 to 1.0, output 
75 values less than 0.5 are normally taken to represent one state such as "false", whereas output values 
greater than 0.5 are taken to represent a complementary state such as "true". The farther from 0.5 an 
output value is, the greater its strength. It is generally desirable to treat a region centered at 0.5 and ranging 
from 0.4 to 0.6, for example, as a buffer zone between "true" and "false", corresponding to a "no 
conclusion" state. 

20 Accordingly, the state of node 302 (Figure 11 A) indicates wire presence and absence. Furthermore, 
each node in panel feature detector 300 has a connection to every node in following layers. 

Figure 11B illustrates a single node 352 which could be any of the nodes illustrated in Fig. 11 A. Each 
node receives input from nodes in the prior layers. 

Weighting elements 340-350 govern the influence of each prior node on the subject node 352. 
25 Each node adds up the weighted node values received from prior nodes as follows: 

N 

30 A= Wo + S Ii x Wi 

i = l 

wherein A is called activation if a node, li is the node value received from other nodes, and Wi is the 
35 weighting factor providing the extent of influence of the prior nodes in the subject node. 

The activation A for each node K is then squashed by the "squashing function" as follows: 

f(A): 1 

40 -A 

1 + e 

In order to train the neural network of Fig. 11 A, a back propagation method may be implemented. 

45 Accordingly, wire pattern may be provided to panel feature detector 300. For a reliable response it is 
desired that the output of node 302 be 0.9 for wire pixels and 0.1 for substrate pixels. 

However, during the training mode, for example, the output of node 302 when initially exposed to wire 
pixels may be less than 0.9. Consequently, an error is generated which is a value between the desired 0.9 
and the actual output of node 302. 

so Thereafter, the input weights to nodes 322-332 and 320-320 and 302 are manipulated such that the 
value of the output at node 302 gets closer to 0.9. The neural net is also exposed to substrate pixels. The 
weights of all the nodes in the network may be manipulated so that the value of the output in node 302 gets 
closer to 1.0, for all substrate pixels. The neural network is exposed to many new wire and substrate pixels 
and adjusted as necessary until all new pixels are consistently classified correctly. 

55 Once the neural network is trained, the system will be ready for inspection mode. 

It can be appreciated by those skilled in the art that the neural network need not to be implemented as 
a fully connected network. Accordingly, a partially connected network wherein some of the weighting 
elements have a zero value may be implemented as well. A zero weighting element causes a node 
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connection to vanish. 

Hue Based Feature Detector 

5 A still further embodiment of the invention is provided by substituting panel feature detector 80 (Fig. 3) 

with panel feature detector 350 as shown in Fig. 12A. 

Panel feature detector 350 receives RGB information from frame grabber 12 (Fig. 3). The panel feature 

detector 350 includes an RGB to HSI converter 352 which receives the RGB values corresponding to each 

pixel being analyzed and converts the RGB value into hue, saturation, and intensity values. 
io Like RGB, an HSI (hue, saturation, intensity) description of a color image uses three components. By 

using HSI components, it is possible to utilize only one component, especially hue, to detect the feature of 

a panel. 

For HSI applications, frame grabber 12 may be a commercially available device that converts RGB 
components to HSI components. 
75 The H component from HSI converter 352 is the provided to threshold adjuster 354. During training 

mode, threshold adjuster 354 sets upper and lower limit parameters T1 and T2 within which a hue value 
represents a wire pixel. 

Figures 12B and 12C illustrate hue threshold parameters T1 356 and T2 358. Thus, for hue values with 
the range of T1 through T2, Fig. 12B, a wire pixel is attributed. 
20 In the HSI domain, hue is expressed as an angle between 0 and 360 degrees. Hue is derived by 

summing the contributions of a Red vector at 0 degrees, a Green vector at 120 degrees, and a Blue vector 
at 240 degrees. Fig. 12B illustrates the hue thresholds T1 and T2 in reference to the Red, Green and Blue 
vectors. Thus, the location of and distance between T1 and T2 can be adjusted until a reliable threshold for 
determining the features of the panel is found. 

25 

Operation 

The operation of the system of the invention includes two separate nodes. The first mode is that of 
training the system so that it can differentiate features on the panel. Specifically, during training mode it is 

30 possible to teach the system to differentiate wires from substrate. 

As illustrated in Figures 1A and 3, a discrete wiring panel 18 is placed on the X-Y table 16 for training 
the system. It is assumed that at this point the system is not capable of differentiating wire features on 
panel 18 from the substrate. Console monitor 50 provides a menu of options for the operator's use. Once 
the discrete wiring panel 18 is placed on the X-Y table 16, the operator may conduct a manual coupon 

35 registration. Microprocessor 10 initializes X-Y transformer 58 with zero offset values. Therefore, actual X-Y 
values are provided to motion system 22 to move X-Y table 16 until a registration coupon appears under 
RGB camera 14. 

Meanwhile, the operator may visually trace the movement of the X-Y table on video monitor 24. 

Fig. 2B illustrates a registration coupon 29 that can be seen on the video monitor 24. A cross-hair 400 

40 is superimposed on the image shown on the video monitor. The cross-hair 400 corresponds to the center of 
the camera lens to aid the operator in aligning the registration coupon. 

Accordingly, X-Y table 16 is moved until the center of the cross-hair 400 fall on the center of 
registration coupon 29. Once the operator determines that a registration coupon is aligned, the X, Y 
coordinates of the center of the registration coupon may then be stored by pressing a button on keyboard 

45 52 (Fig. 3). This operation is repeated until X, Y coordinates of all four registration coupons have been 
determined. The X, Y coordinates are then provided to compensator 72 (Fig. 3) so that the actual position of 
registration coupons can be compared with their reference positions stored in panel file 55 (Fig. 3). A 
separate panel file 55 is used to describe the coupon arrangements for all the various panel types. 
Compensator 72 then provides the proper translation matrix to X, Y transformer 58. 

so Once the panel has been manually registered, the operator may begin the training mode by pressing 
another button on the keyboard 52. Microprocessor 10 then superimposes a training rectangle 402 (Fig. 1) 
on the image retrieved by RGB camera 14. The dimension of the training rectangle 402 may be provided 
algorithmically, or by a training rectangle table 93. According to one embodiment of the invention, the 
length and width of the training rectangle is 2 x 2 mils, which encloses a ten by ten pixel area. Training 

55 rectangle 402 can be superimposed over any desired portion of frame grabber 12. Video monitor 24 
displays the image of rectangle 402 moving over displayed image. 

The operator then moves the training rectangle 402 such that it encloses a piece of a substrate scene. 
When this is accomplished, the operator instructs the system to sample the RGB values within this training 
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rectangle 402. 

Upon receiving "sample" instruction, microprocessor 10 provides a signal to frame grabber 12 to freeze 
the image being retrieved by RGB camera 14. Training rectangle table 93 provides the information 
necessary to retrieve the pixels enclosed in the training rectangle area to pixel row buffer 88. Since the 

5 RGB values retrieved correspond to a substrate region, only the bits in the substrate look-up table 
associated with the RGB values will be set to "1". Since the digitized video signals contain a significant 
amount of noise, it is advantageous to sample the pixels within the rectangle a number of times. In one 
embodiment of the invention, each time the operator initiates data sampling, all pixels in the rectangle are 
samples twenty times. This produces two thousand samples per operator action. As a result, the relevant 

w regions of the look-up table 84 corresponding to the substrate will fill with "ones" quickly. The operator 
continues to take samples of more substrate scenes. Each time sampling is complete, the areas of the 
displayed image now known to be the substrate according to the substrate look-up table become artificially 
tinted green. This provides the operator with the feedback as to how well the training is progressing. During 
sampling of the substrate, the operator attempts to capture as many variations as possible. Every nuance of 

75 substrate which is captures is then guaranteed not to be mistaken as wire after training, It is also desirable 
that the light intensity and focus be varied during training. This incorporates light and focus variations into 
panel feature detector 80. 

Thereafter, the operator starts placing scenes of wire in the training rectangle 402 and instructs the 
system to sample and store the data in the wire look-up table. Again, the operator could conduct the 
20 training under fluctuating light and focus conditions. Features extraneous to wire are prevented from being 
incorporated into the wire look-up table 82 by keeping them out of the training rectangle 402. This time, all 
areas of the image on the video monitor 24 with RGB values corresponding to wire look-up table 82 
become tinted artificially red each time sampling is complete and all areas corresponding to substrate get 
tinted green again as well. 

25 Inhibitor 86 (Fig. 3) compares the RGB values in both look-up tables 82 and 84. In the event that an 

RGB value has set a bit in both substrate look-up table 84 and wire look-up table 82, the bit in the wire 
look-up table is set back to zero. Accordingly, the RGB value will be interpreted as substrate and the pixel 
corresponding to the RGB value becomes tinted green. 

The recognition accuracy in the end is a function of the diversity of scene characteristics to which the 

30 system has been exposed during training. To increase the efficiency of training, the operator has the ability 
to dilate the substrate look-up table 84 and wire look-up table 82. Under dilation, any bits with value "one" 
force their immediate neighbors in all three dimensions to become "one" as well. This has the effect of 
forcing generalization. Image regions which by virtue of lack of artificial tinting are observed to be undefined 
or dotted with noise, quickly change to a defined state under dilation. Finally, when all regions of additional 

35 novel images are strongly and correctly classified, training is complete. The operator instructs the system to 
store the 32k byte wire look-up table 82 onto RGB file 60 (Fig. 3) which, in one embodiment, is a computer 
disk. 

Figures 12A-12C illustrate the training operation of the system when panel feature detector 350 (Fig. 
12A) is substituted for the panel feature detector 80 (Fig. 3). Accordingly, thresholds 356 and 358 may be 

40 determined and set by utilizing video monitor 24 (Figs. 1 and 3). The circular representation of the RGB 
vectors (Fig. 12B) can be represented by a linear hue spectrum 360 (Fig. 12C). Thus, hue values 
coresponding to RGB vectors can be represented by a number between 0 and 250 on hue spectrum 360. 

A white bar 362 is superimposed on the image shown on video monitor 24. The operator can move the 
white bar 362 side to side along the hue spectrum 360. The operator can also shorten or enlarge the white 

45 bar. Thus, the thresholds Ti and T 2 on the hue spectrum may be defined (Fig. 12B). 

Consequently, all pixel element having hue values within the boundary defined by white bar 362 are 
associated with wire, and pixel elements having hue values outside the boundary defined by white bar 362 
are associated with substrate features. 

The training of the neural network feature detector is similarly conducted as explained above. 

so Once the system is trained, it is capable of inspecting similar wire scribed circuit panels. Accordingly, 
the operator places a circuit panel 18 to be inspected on an X, Y table 16. Microprocessor 10 retrieves the 
coordinates of registration coupons one at a time from the panel file. The coordinates of the registration 
coupon are then provided to X, Y table 16 so that the table is moved such that the vicinity of the registration 
coupon is placed under RGB camera 14. Once a registration coupon is encountered by the RGB camera, 

55 coupon registration 62 is invoked to locate the center of the registration coupon. The coordinates of the 
located center thus become the actual coordinates which are compared with reference coordinates. 
Accordingly, the motion controller moves the X, Y table 16 in both the X and Y coordinates so that row 
profile buffer 64 and column profile buffer 66 can receive wire pixel counts corresponding to three horizontal 
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and vertical lines of the registration coupon. Registration 62 finally derives the centers of the coupons and 
provides the information for the transformation matrix in X, Y transformer 58. 

Thereafter, inspection of the hole sites begins by retrieving the coordinates of each hole site to be 
inspected from job file 54. The coordinates of each hole site are then fed to X, Y transformer 58 so that the 
5 actual hole site on a particular panel being inspected can be located. 

Once a hole site is positioned under the RGB camera, open/short scan table 96 and keepout scan table 
98 provide corresponding data so that the appropriate pixels for analysis are retrieved and sent to panel 
feature detector 80. Thereafter, short detector 100 determines whether the wire crossings on the hole site 
boundary are violating any rules explained before. Specifically, short detector 100 determines the number of 
w wire crossings over the hole site boundary and the locations of entry and exit. 

Similarly, open detector 120 determines whether the wire is placed accurately enough that a substantial 
part of it is within the open reticle. 

For hole sites where no wire crossings are allowed, the keepout detector determines whether any wire 
has illegally entered the hole site. The system then analyzes all the hole sites in the wiring panel 18 one at 
75 a time. Job file 54 preferably is formed such that the distances travelled in visiting all the points is 
minimized by applying a point order optimization phase. The optimization is done on a per hole-class basis. 
When traversing the points in an optimized inspection job file, all points within a given hole class are 
encountered together. This is because computational overhead is involved when the inspection system 
adjusts for a new hole code. 

20 As explained before, every time a new hole code is to be utilized, pixel/bin map 90 has to be initialized 

which involves computational overhead. The hole code changes that are required during inspection are, 
therefore, intentionally minimized during optimization. 

Every time a defect is determined, the information corresponding to the hole site is stored in a defect 
file 56. Once all the hole sites have been inspected, the system goes to review mode, wherein the operator 

25 can visually inspect all hole sites having apparent defects. 

The locations of defective hole sites are retrieved from defect file 56 for manual inspection. The 
operator the determines whether errors flagged by the system are significant or not and whether they can 
be easily by corrected. Thus, the task of inspecting many hole sites becomes increasingly reliable and 
simple. 

30 Figures 13A-13F illustrate some examples of wire crossings at hole sites. Accordingly, Figure 13A 

illustrates a defective wire termination, since wire 282 does not end sufficiently within the open circuit reticle 
164. Therefore, it is likely that when the hole site is drilled, an open circuit occurs for lack of connection 
between wire 282 and the conductive plating to enter the drilled hole site. Conversely, Figure 13D illustrates 
a proper wire termination having sufficient mass within the open circuit reticle 164. Similarly, Figures 13B 

35 and 13E illustrate a defective wire inflection and a desired wire inflection. Figure 13C illustrates a defective 
hole site which can lead to a possible short circuit occurrence when the hole is drilled. In the example 
illustrated by Figure 13E, the short circuit detection will detect more that two wire crossings due to illegal 
wire 283 crossing the hole site. Figure 13E illustrates the desired situation for wire crossings. 

It can be appreciated by those skilled in the art that the system of the invention may be easily modified 

40 to perform inspection of printed circuit boards. Consequently, the system may be trained to differentiate 
between the conductor traces on the printed circuit board and the substrate over which the conductor traces 
are disposed. The system of the invention is particularly advantageous in situations where the light intensity 
reflected from the conductor trace and the substrate is very similar. Conventional black and white imaging 
systems as mentioned before become ineffective when the light intensity reflected from the conductor trace 

45 is similar to the light intensity reflected from the substrate. 

Accordingly, for printed circuit board inspection, appropriate design rules commonly known in the art 
may be implemented such that conductor traces comply with certain minimum design requirements. For 
example, threshold tolerances may be set for the width of conductors or the gap between two conductors 
below which the printed circuit board would not be acceptable. 

so As illustrated and described above, the invention can be implemented by various embodiments. The 
invention in its broader aspects is, therefor, not limited to the specific embodiments herein shown and 
described but departures may be made therefrom within the scope of the accompanying claims without 
departing from the principles of the invention and without sacrificing its chief advantages. 

55 Claims 

1. An optical inspection system for detecting faults at hole sites of a wire scribed circuit board (18) having 
a plurality of wires (28) scribed on a substrate, said system comprising 
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a video camera (14) for receiving images of desired holes sites (30) on said wire scribed circuit 
board (18), said video camera (14) converting each image to electrical video signals; 

a frame grabber (12) for converting said electrical video signals from said video camera (14) to a 
plurality of pixels and for storing information representing color element values contained in each of 
5 said pixels; 

a panel feature detector (80, Fig. 3) receiving said color element values of a pixel to be analyzed 
from said frame grabber (12), providing a wire indication when said color element values of said pixel 
correspond to a wire feature; 

a pixel retrieve controller (91) providing command signals to said frame grabber (12) to transmit to 
w said panel feature detector (80) color element values corresponding to predetermined pixels associated 

with a hole site to be analyzed; 

an open detector circuit (120) receiving said wire indication from said panel feature detector (80) 
corresponding to said predetermined pixels associated with said hole site, said open detector circuit 
(120) determining whether an open circuit could occur when said hole site is drilled. 

/5 

2. The system of claim 1 further comprising a short detector circuit (100) receiving said wire indication 
from said panel feature detector (80) corresponding to said predetermined pixels associated with said 
hole site, said short detector circuit (100) determining whether a short circuit could occur when said 
hole site is drilled. 

20 

3. The system of claim 1 further comprising a keepout detector circuit (130) receiving said wire indication 
from said panel feature detector (80) corresponding to said predetermined pixels associated with said 
hole site, said keepout detector circuit (130) determining whether no wire is crossing said hole site. 

25 4. The system of claim 3 wherein said frame grabber (12) provides data in a scan line basis to a pixel row 
buffer (88) retaining information of all pixels contained on a scan line and providing said information to 
said panel feature detector (80) on a pixel by pixel basis. 

5. The system of claim 4 wherein said pixel retrieve controller (91) further comprises 

30 an open/short scan table associated with a hole site having a plurality of corresponding scan lines, 

said open/short scan table for each said scan line storing 
an open/short scan line number, 

the number of pixels from a vertical diameter crossing through said hole site to the outer and inner 
circles formed by a short circuit reticle, and 
35 the number of pixels from said vertical diameter crossing through said hole site to a circle formed 

by an open circuit reticle; 

a keepout scan table associated with a hole site having a plurality of corresponding scan lines, said 
keepout scan table for each said scan line storing 
a keepout scan line number, and 
40 the number of pixels from a vertical diameter crossing through said hole site to the circle formed 

by a keepout reticle; 

a pixel row transfer controller receiving said open/short scan line number and said keepout scan 
line number to transfer corresponding scan lines from said frame grabber to said pixel row buffer; and 
a pixel row scan controller receiving said number of pixels from said open/short scan table and said 
45 keepout scan table for directing corresponding pixels from said pixel row buffer to said panel feature 

detector. 

6. The system of claim 5 wherein said shorts detector (100) further comprises 

a bin histogram (102) having a plurality of bins dividing said short circuit reticle into equal parts, 
so each bin having a corresponding bin number, said bin histogram (102) storing the number of detected 

wire pixels associated with each bin number; 

a digital filter (104) for processing data stored in said bin histogram (102) for identifying the peak 
occurrences of said detected wire pixels; and 

an analyzer for determining whether the ratio of the number of wire pixels detected in the highest 
55 two peak occurrences over the total number of pixels in said bin histogram is above a predetermined 

threshold. 
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7. The system of claim 6 wherein said open detector (120) further comprises 

a wire pixel counter (122); and 

a comparator (124) for determining whether the number of wire pixels detected in an open circuit 
reticle is above a predetermined threshold. 

5 

8. The system of claim 7 wherein said keepout detector (120) further comprises 

a wire pixel counter (132); and 

a comparator (134) for determining whether the number of wire pixels detected in a keepout reticle 
is less than a predetermined threshold. 

w 

9. The system of claim 8 further comprising a registration analyzer (62) comprising 

row profile butter (64) receiving said wire pixel indication from said panel feature detector (80); 
a column profile buffer (66) receiving said wire pixel indication from said panel feature detector 
(80); 

75 a cluster identifier (68) for recognizing significant wire pixel peak occurrences in said row and said 

column profile buffers (64, 66); 

a cluster analyzer (70) for determining the X-Y coordinates of the center of said wire pixel peak 
occurrences corresponding to data in said row profile buffer (64) and said column profile buffer (66); 
and 

20 a compensator (72) for comparing the desired location of said center and actual location of said 

center determined by said cluster analyzer (70) and computing a transformation matrix so that errors 
arising from an improper alignment of said wire circuit board (18) can be compensated. 

10. The system of claim 9 wherein said panel feature detector (80) further comprises 
a wire look-up table (82) representing a three dimensional RGB space addressed by a number 

corresponding to the magnitude of color element values of a pixel provided by said frame grabber (12), 
the data corresponding to said address set to "1" when said color element values of said pixel 
correspond to a wire pixel; 

a substrate look-up table (84) representing a three dimensional RGB space addressed by a number 
corresponding to the magnitude of color element values of a pixel provided by said frame grabber (12), 
the data corresponding to said address set to "1" when said color element values of said pixel 
correspond to a substrate pixel; and 

an inhibitor (86) for setting an addressed bit in said wire look-up table (82) back to zero whenever 
the same color element values of a pixel have set an addressed bit in both wire and substrate look-up 
tables to "1 ". 

11. The system of claim 10 further comprising a dilator locating said bits in said wire and substrate look-up 
tables with a value "one" and setting immediate neighbors of said set bits in all three dimensions of 
said RGB space corresponding to said wire and substrate took-up tables with a value "1". 

12. The system of claim 9 wherein said panel feature detector (80) further comprises 
an RGB to HSI converter receiving the RGB values corresponding to each pixel being analyzed 

and providing a hue component corresponding to said pixel being analyzed; and 

a threshold adjuster having a first and a second hue parameter defining a range of hue values such 
that said pixel being analyzed is attributed to a wire pixel when the value of said hue component 
corresponding to said pixel being analyzed is within said range of hue values. 

13. The system of claim 9 wherein said panel feature detector (80) further comprises 
a neural network having an input layer containing three nodes with linear transfer functions for 

distributing incoming RGB data corresponding tp a pixel to be analyzed to the rest of said neural 
network; 

a plurality of hidden layers containing a plurality of nodes with sigmoidal transfer functions, each 
node having connections to predetermined nodes in following layers; and 

an output node indicating whether said pixel to be analyzed is a wire pixel or a substrate pixel. 

14. The system of claim 13 wherein said neural network is a two hidden layer fully connected feed forward 
network. 
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15. An optical inspection system for differentiating wires from substrate in a wire scribed circuit board 
having a plurality of wires scribed on said substrate, said system comprising 

a video camera for receiving desired images of said wires scribed circuit board, said video camera 
converting each image to electrical video signals; 
s conversion means for converting said electrical video signals into a plurality of scan lines having a 

plurality of pixels; 

a memory means for storing information representing RGB color element values contained in each 
of said pixels of said images; 

a wire look-up table and a substrate look-up table each of said tables having a plurality of bytes; 
w said wire look-up table and said substrate look-up table having an address by combining binary values 
representing each component of said RGB color element values, wherein at least three significant bits 
of said address point to a bit location on one of said bytes and remaining most significant bits of said 
address point to a byte location. 

/5 16. The system of claim 15 wherein all RGB values that belong to wire features will address a "one" bit in 
said wire look-up table and all RGB values that belong to a substrate feature will address a "one" bit in 
said substrate look-up table. 

17. The system of claim 16 wherein an addressed bit in said wire look-up table is set back to "zero" 
20 whenever the same RGB values corresponding to a pixel have set an addressed bit in said substrate 

look-up table to "one". 

18. The system of claim 17 wherein each dimension of the RGB space corresponding to Red, Green and 
Blue colors has a resolution of 6 bits. 

25 

19. A method for detecting faults at hole sites of a plurality of wire scribed circuit boards each having a 
plurality of wires scribed on a substrate, said method comprising the steps of 

receiving desires images from each wire scribed circuit board; 

converting each of said desired images to a plurality of wire and substrate pixels, said wire pixels 
30 corresponding to said wires scribed on said substrate and said substrate pixels corresponding to said 

substrate; 

displaying said desired images received from said wire scribed circuit board; 

deriving color element values contained in each of said wire and substrate pixels of each image; 

providing a training mode for a sample wire scribed circuit board by 
35 associating color element values of said wire pixels to a wire indication and associating color 

element values of said substrate pixels to a substrate indication; 

tinting all pixels in each displayed image attributed to wire pixel with a first color; 

tinting all pixels in each displayed image attributed to substrate pixel with a second color; 

providing an inspection mode for each of said plurality of wire scribed circuit boards by 
40 retrieving color element values corresponding to predetermined pixels associated with a hole site to 

be analyzed and providing said wire indication when said color element values correspond to a wire 
pixel; 

analyzing said wire indication corresponding to said predetermined pixels to determine whether an 
open circuit could occur when said hole site is drilled. 

45 

20. The method of claim 19 further comprising the step of analyzing said wire indication corresponding to 
said predetermined pixels to determine whether short circuit could occur when said hole site is drilled. 

21. The method of claim 20 further comprising the step of analyzing said wire indication corresponding to 
so said predetermined pixels to determine whether a wire is crossing said hole site. 

22. The method of claim 21 (19) wherein said step of converting further comprises the step of receiving 
each one of said desired images in a scan line basis; and retaining information of all wire and substrate 
pixels contained in a scan line. 

55 

23. The method of claim 22 (19) wherein said step of retrieving color element values further comprises the 
steps of 

providing an open/short scan table associated with a hole site having a plurality of scan lines, said 
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open/short scan table for each hole site providing the identity of scan lines to be received and the 
identity of pixels to be analyzed in open and short circuit reticles; 

providing a keepout scan table associated with said hole site having a plurality of corresponding 
scan lines, said keepout scan table for each hole site providing the identity of scan lines to be received 
5 and the identity of pixels to be analyzed in a keepout reticle. 

24. The method of claim 23 (19) wherein said step of analyzing said wire indication further comprises the 
steps of 

providing a bin histogram having a plurality of bins dividing said short circuit reticle into equal 
10 parts, each bin having a corresponding bin number, said bin histogram storing the number of detected 

wire pixels associated with each bin number; 

processing data stored in said bin histogram for identifying the peak occurrences of said detected 
wire pixels; and 

determining whether the ratio of the number of wire pixels detected in the highest two peak 
is occurrences to the total number of wire pixels detected in said bin histogram exceeds a predetermined 

threshold. 

25. The method of claim 23 further comprising the steps of 

setting appropriate locations in a wire look-up table addressed by color element values associated 
20 with a wire pixel to a "1 " denoting wire indication; 

setting appropriate locations in a substrate look-up table addressed by color element values 
associated with a substrate pixel to "1" denoting substrate indication; and 

inhibiting an addressed bit in said wire look-up table back to zero whenever the same color 
element values of a pixel have set an addressed bit in both wire and substrate look-up tables to "1". 

25 

26. The method of claim 23 wherein said step of deriving color element values further comprises the step 
of 

providing a hue component corresponding to each pixel being analyzed; and 

attributing said pixel to be analyzed to a wire pixel when said hue component is within a 
30 predetermined range of hue values defined by a first and second hue parameter. 

27. The method of claim 23 wherein said step of deriving color element values further comprises the step 
of 

providing a neural network having an input layer containing three nodes with linear transfer 
35 functions for distributing incoming RGB data corresponding to a pixel to be analyzed to the rest of said 

neural network; 

generating a plurality of hidden layers containing a plurality of nodes with sigmoidal transfer 
functions, each node having connections to predetermined nodes in following layers; 

providing an output node indicating whether said pixel to be analyzed is a wire pixel or a substrate 
40 pixel. 

28. The method of claim 27 wherein said step of generating further comprises generating a neural network 
being two hidden layer fully connected feed forward network. 

45 29. An optical inspection system for detecting faults at desired inspection sites of a printed circuit board 
having a plurality of conductor traces disposed on a substrate, said system comprising 

a video camera for receiving images of desired inspection sites on said printed circuit board, said 
video camera converting each image to electrical video signals; 

a frame grabber for converting said electrical video signals from said video camera to a plurality of 
so pixels and for storing information representing color element values contained in each of said pixels; 

a panel feature detector receiving said color element values of a pixel to be analyzed from said 
frame grabber, providing a conductor trace indication when said color element values of said pixel 
correspond to a conductor trace; 

a pixel retrieve controller providing command signals to said frame grabber to transmit to said 
55 panel feature detector color element values corresponding to predetermined pixels associated with said 

desired inspection site to be analyzed; 

a fault detector circuit receiving said conductor trace indication from said panel feature detector 
corresponding to said predetermined pixels associated with said inspection site, said fault detector 
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determining whether a fault is present at said desired inspection site. 

30. The system of claim 29 wherein said fault detector determines the gap between two conductor traces 
and compares said gap with a predetermined threshold value. 

5 

31. The system of claim 30 wherein said fault detector further determines the width of a conductor trace to 
be inspected and compares said width with a predetermined threshold value. 

32. The system of claim 31 wherein said frame grabber provides data on a scan line basis to a pixel row 
io buffer retaining information of all pixels contained in a scan line and providing said information to said 

panel feature detector on a pixel by pixel basis. 

33. The system of claim 32 further comprising a registration analyzer comprising 

a row profile butter receiving said conductor trace pixel indication from said panel feature detector; 
75 a column profile buffer receiving said conductor trace pixel indication from said panel feature 

detector; 

a cluster identifier for recognizing significant conductor trace pixel peak occurrences in said row 
and said column profile buffers; 

a cluster analyzer for determining the X-Y coordinates of the center of said conductor trace pixel 
20 peak occurrences corresponding to data in said row profile buffer and said column profile buffer; and 

a compensator for comparing the desired location of said center and actual location of said center 
determined by said cluster analyzer and computing a transformation matrix so that misalignment errors 
of said printed circuit board can be compensated. 

25 34. The system of claim 33 wherein said panel feature detector further comprises 

a conductor trace look-up table representing a three dimensional RGB space, said conductor trace 
look-up table addressed by a number corresponding to the magnitude of color element values of a 
pixel provided by said frame grabber, the data corresponding to said address set to "1" when said 
color element values of said pixel correspond to a conductor trace pixel; 

30 a substrate look-up table representing a three dimensional RGB space addressed by a number 

corresponding to the magnitude of color element values of a pixel provided by said frame grabber, the 
data corresponding to said address set to "1" when said color element values of said pixel correspond 
to a substrate pixel; and 

an inhibitor for setting an addressed bit in said conductor trace look-up table back to zero 
35 whenever the same color element values of a pixel have set an addressed bit in both conductor trace 

and substrate look-up tables to "1". 

35. The system of claim 34 further comprising a dilator locating said bits in said conductor trace and 
substrate look-up tables with a value "one" and setting immediate neighbors of said set bits in all three 

40 dimensions of said RGB space corresponding to said conductor trace and substrate look-up tables with 

a value "1 

36. The system of claim 33 wherein said panel feature detector further comprises 

an RGB to HSI converter receiving the RGB values corresponding to each pixel being analyzed 
45 and providing a hue component corresponding to said pixel being analyzed; and 

a threshold adjuster having a first and a second hue parameter defining a range of hue values such 
that said pixel being analyzed is attributed to a conductor trace pixel when the value of said hue 
component corresponding to said pixel being analyzed is within said range of hue values. 

so 37. The system of claim 33 wherein said panel feature detector further comprises 

a neural network having an input layer containing three nodes with linear transfer functions for 
distributing incoming RGB data corresponding to a pixel to be analyzed to the rest of said neural 
network; 

a plurality of hidden layers containing a plurality of nodes with sigmoidat transfer functions, each 
55 node having connections to predetermined nodes in following layers; and 

an output node indicating whether said pixel to be analyzed is a conductor trace pixel or a 
substrate pixel. 
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38. The system of claim 37 wherein said neural network is a two hidden layer fully connected feed forward 
network. 
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CLAIMS INCURRING FEES 



The present European patent application comprised at the time o< filing more than ten ctalms. 

| 1 A " claim3 1w °«« n P«W **^n the prescribed time Mm it The present Europe** sea/cr. report has been 

drawn up for alt claims. 

| | 0o,v Dart 01 lne cla«ms tees have been paid within the prescribed time limit The present European search 
report has been drawn up lor tne first ten claims and (or those claims for which claims fees have been paid. 

namely claims: 

I I No clairn s have been paid within the prescribed time limit. The present European search repon has been 
drawn up tor the first ten claims. 



LACK OF UNITY OF INVENTION 



The Search Division considers that the present European patent application does not compty with tne requirement of unity of 
invention and relates to several Inventions or groups of Inventions. 

namely: 



2. 



Claims 1-14,19-38: Color element value retrieval for 
inspection site pixels in optical circuit board 
inspection system. 

Cairns 15-18: Address organization of colour look-up 
tables in optical circuit board inspection systems. 



r~-| AH further search fees have been paid within the li.ed i.me Um,t The present European search repon has 
been drawn up for all claims. 

r-j Oni y par1 of tn « , urtn<! r search tees have been pa.o w.mm tne )u«3 time lim.t. The present European search 
revon has Been drawn up for mose parts ol the European paient appi.catton wn.cn reiaie to Ihe Inventions in 
respect of which search fees have been pe«d. 

namely clatmi; 

None ot me further search fees has been paid w-min me f,-eo t.m« um.t. The present Eurooean search repon 
n« oeen drawn up lor mose parts ol tne Eufop«an paion. eopxcauon wrwch re.ate to ih« lnvenit on first 

mentioned in me ciam-.j. 
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